Vehicle fault detection system and method utilizing graphically converted temporal data

ABSTRACT

A vehicle fault detection system including at least one sensor configured for coupling with a vehicle system, a vehicle control module coupled to the at least one sensor, and being configured to receive at least one time series of numerical sensor data from the at least one sensor, at least one of the at least one time series of numerical sensor data corresponds to a respective system parameter of the vehicle system being monitored, generate a graphical representation for the at least one time series of numerical sensor data to form an analysis image of at least one system parameter, and detect anomalous behavior of a component of the vehicle system based on the analysis image, and a user interface coupled to the vehicle control module, the user interface being configured to present to an operator an indication of the anomalous behavior for the component of the vehicle system.

BACKGROUND 1. Field

The exemplary embodiments generally relate to fault detection and inparticular to fault detection by graphically converting temporal data.

2. Brief Description of Related Developments

Generally, fault detection in vehicles such as aircraft is performedusing some form of statistical analysis. Generally digital sensor datais obtained in a time series of sensor data and is converted into amathematical form for statistical (or other) processing using, forexample, machine learning based solutions. These machine learning basedsolutions extract statistical measures, known as features, from adataset, such as the time series of sensor data. Examples of thefeatures include a minimum, a maximum, or an average parameter valueover the course of an entire vehicle excursion (which in the case of anaircraft is an entire flight). Values for the features are comparedacross a series of vehicle excursions in an attempt to identify a trendin the time series of sensor data that precedes a vehicle componentfault.

Generally, the features being analyzed are manually defined, which maybe very time consuming. Further, the dataset that makes up the timeseries of sensor data is composed of tens of thousands of sensor values.With statistical analysis of the time series of sensor data, the entiredataset generally gets reduced or summarized into a single number. Assuch, conventional statistical vehicle fault detections systems mayignore large volumes of data and may not be able to capture subtlechanges in the data or complex patterns inherent to the data (e.g.,which may include relationships between vehicle components with respectto faults).

SUMMARY

Accordingly, apparatuses and methods, intended to address at least oneor more of the above-identified concerns, would find utility.

The following is a non-exhaustive list of examples, which may or may notbe claimed, of the subject matter according to the present disclosure.

One example of the subject matter according to the present disclosurerelates to a vehicle fault detection system comprising: at least onesensor configured for coupling with a vehicle system; a vehicle controlmodule coupled to the at least one sensor, the vehicle control modulebeing configured to receive at least one time series of numerical sensordata from the at least one sensor, at least one of the at least one timeseries of numerical sensor data corresponds to a respective systemparameter of the vehicle system being monitored, generate a graphicalrepresentation for the at least one time series of numerical sensor datato form an analysis image of at least one system parameter, and detectanomalous behavior of a component of the vehicle system based on theanalysis image of at least one system parameter; and a user interfacecoupled to the vehicle control module, the user interface beingconfigured to present to an operator an indication of the anomalousbehavior for a component of the vehicle system.

Another example of the subject matter according to the presentdisclosure relates to a vehicle fault detection system comprising: amemory; at least one sensor coupled to the memory, the at least onesensor being configured to generate at least one time series ofnumerical sensor data for a respective system parameter of a vehiclesystem being monitored; a vehicle control module coupled to the memory,the vehicle control module being configured to transform the at leastone time series of numerical sensor data for the respective systemparameter into an analysis image of at least one system parameter anddetect, with at least one deep learning model, anomalous behavior of therespective system parameter based on the analysis image of at least onesystem parameter; and a user interface coupled to the vehicle controlmodule, the user interface being configured to present to an operator anindication of the anomalous behavior of the respective system parameter.

Still another example of the subject matter according to the presentdisclosure relates to a method for vehicle fault detection, the methodcomprising: generating, with at least one sensor coupled to a vehiclesystem, at least one time series of numerical sensor data for arespective system parameter of the vehicle system being monitored;transforming, with a vehicle control module coupled to the at least onesensor, the at least one time series of numerical sensor data for therespective system parameter into an analysis image of at least onesystem parameter; detecting, with at least one deep learning model ofthe vehicle control module, anomalous behavior of the respective systemparameter based on the analysis image of at least one system parameter;and displaying, on a user interface coupled to the vehicle controlmodule, an indication of the anomalous behavior of the respective systemparameter.

BRIEF DESCRIPTION OF THE DRAWINGS

Having thus described examples of the present disclosure in generalterms, reference will now be made to the accompanying drawings, whichare not necessarily drawn to scale, and wherein like referencecharacters designate the same or similar parts throughout the severalviews, and wherein:

FIG. 1 is a schematic block diagram of a vehicle fault detection systemin accordance with aspects of the present disclosure;

FIG. 2 is a schematic illustration of converting a time series ofnumerical sensor data to an image in accordance with aspects of thepresent disclosure;

FIGS. 3A-3D are illustrations of exemplary images generated from one ormore time series of numerical sensor data, where each image includesdifferent number system parameters in accordance with aspects of thepresent disclosure;

FIGS. 4A-4C are exemplary images of respective vehicle excursions inaccordance with aspects of the present disclosure;

FIG. 5 is an exemplary illustration of the division of the image of FIG.4A into one or more temporal sub-regions in accordance with aspects ofthe present disclosure;

FIGS. 6A-6C are exemplary images of events that may occur within theimages of FIGS. 4A-C and 5 in accordance with aspects of the presentdisclosure;

FIG. 7A is a schematic illustration of a deep learning model inaccordance with aspects of the present disclosure;

FIG. 7B is a schematic illustration of a deep learning model inaccordance with aspects of the present disclosure;

FIG. 8 is an exemplary illustration of an output of the deep learningmodel of FIG. 7A in accordance with aspects of the present disclosure;

FIG. 9A is an exemplary illustration of an output of the deep learningmodel of FIG. 7B in accordance with aspects of the present disclosure;

FIG. 9B is an exemplary illustration of an output of the deep learningmodel of FIG. 7B in accordance with aspects of the present disclosure;

FIG. 9C is an exemplary illustration of a combined output of the deeplearning model of FIG. 7B in accordance with aspects of the presentdisclosure;

FIG. 10 is an exemplary flow diagram for training a deep learning modelin accordance with aspects of the present disclosure;

FIG. 11 is an exemplary flow diagram for determining faults in a vehiclesystem in accordance with aspects of the present disclosure;

FIG. 12 is an exemplary illustration of the vehicle in FIG. 1 inaccordance with aspects of the present disclosure; and

FIG. 13 is an exemplary flow diagram of an aircraft production andservice methodology.

DETAILED DESCRIPTION

Referring to FIG. 1, the aspects of the present disclosure provide for asystem 199 and method 1100 (see FIG. 11) for determining vehicle system102 faults that avoids a difficult challenge of reducing at least onetime series of numerical sensor data 112TA-112Tn from one or morevehicle 100 sensors 101 (e.g., a digital signature of a vehiclecomponent) into a mathematical form for statistical or other analysis.The aspects of the present disclosure convert at least one time seriesof numerical sensor data 112TA-112Tn from a vehicle 100 (e.g., such asflight data from an aircraft) into at least one analysis image 180. Theaspects of the present disclosure apply any suitable deep learningmodels 122M to the analysis images 180 to detect temporal anomalies inthe at least one time series of numerical sensor data 112TA-112Tn andpredict an occurrence of an impending vehicle 100 component 102Cfailure. For example, the at least one time series of numerical sensordata 112TA-112Tn from the vehicle 100 are converted to at least one(static) analysis image 180. Fault signatures that precede a vehicle 100component 102C failure are identified from the graphicalrepresentation(s) (embodied in the analysis images 180) of at least onethe time series of numerical sensor data 112TA-112Tn. The aspects of thepresent disclosure may provide for the capability to analyze all (or oneor more) sensor readings in, for example, a vehicle 100 excursion 170,instead of conventional statistical summaries of the vehicle 100excursions 170 (e.g., such as minimum, maximum, or average systemparameter values).

The aspects of the present disclosure provide for the creation ofvehicle prognosis that may not be possible with conventional statisticalfault detection methods, and may increase the accuracy of existingpredictive maintenance solutions (e.g., maintenance schedules, etc.).The aspects of the present disclosure may provide for entire vehicle 100excursions 170 (or at least a portion thereof) to be analyzed so as tofind anomalies in the at least one time series of numerical sensor data112TA-112Tn that conventional (e.g., statistical) fault detectionmethods are unable to detect. The aspects of the present disclosure mayprovide for detection of anomalies that are rooted in complex systemparameter 112A-112 n relationships. The aspects of the presentdisclosure also may eliminate a need for manual feature generation suchas is done with conventional statistical fault detection methods. Theaspects of the present disclosure may also provide a picture of thebehavior that is identified as being anomalous which may helpmaintenance personnel and/or vehicle operators understand and/or believethe fault predictions 189P made by the system 199 and method 1100 (seeFIG. 11) described herein.

Illustrative, non-exhaustive examples, which may or may not be claimed,of the subject matter according to the present disclosure are providedbelow.

Still referring to FIG. 1 and also to FIG. 12, the vehicle faultdetection system 199 will be described with respect to a fixed wingaircraft, such as aircraft 100A for exemplary purposes only. However, itshould be understood that the vehicle fault detection system 199 may bedeployed in any suitable vehicle 100, including but not limited toaerospace vehicles, rotary wing aircraft, fixed wing aircraft, lighterthan air vehicles, maritime vehicles, and automotive vehicles. In oneaspect, the vehicle 100 includes one or more vehicle systems 102 eachhaving respective components 102C (e.g., engines and components thereof,air conditioning systems and components thereof, etc.). The vehiclesystems 102 may include propulsion systems 1210, hydraulic systems 1228,electrical systems 1226, main landing gear systems 1220, and noselanding gear systems 1221. The vehicle 100 may also include an interior1251 having an environmental system 1252. In other aspects, the vehiclesystems 102 may also include one or more control systems coupled to anairframe 1240 of the aircraft 100A, such as for example, flaps,spoilers, ailerons, slats, rudders, elevators, and trim tabs.

Referring to FIG. 1, the vehicle fault detection system 199 includes atleast one sensor 101 configured for coupling with a vehicle system 102.A vehicle control module 110 is coupled to the at least one sensor 101in any suitable manner, such as through any suitable wired or wirelessconnection. The vehicle control module 110 may be any suitablecontroller onboard the vehicle 100 or any suitable controller that iswirelessly coupled to or hardwired to the vehicle 100 (e.g., such as avehicle maintenance controller). The vehicle control module 110 mayinclude any suitable memory 111 and processor 120 configured with anysuitable data storage and non-transitory computer program code forcarrying out the aspects of the present disclosure as described herein,where for example, the at least one sensor 101 is coupled to the memory111 so that data from the at least one sensor 101 is stored in thememory 111 as described herein. The vehicle fault detection system 199may also include any suitable user interface 125 coupled to the vehiclecontrol module 110. The user interface 125 may be a display/interface ofthe vehicle 100 or a display/interface coupled to the vehicle 100through a wired or wireless connection. The user interface is configuredto present to an operator of the vehicle 100 an indication of anomalousbehavior 189 for a component 102C (and/or for a respective systemparameter 112A-112 n of one or more components 102C) of the vehiclesystem 102.

The at least one sensor 101 is configured to generate at least one timeseries of numerical sensor data 112TA-112Tn for a respective systemparameter 112A-112 n of a vehicle system 102 (or component 102C thereof)being monitored. The vehicle control module 110 is configured to receivethe at least one time series of numerical sensor data 112TA-112Tn fromthe at least one sensor 101, such as over the wired or wirelessconnection so that the at least one time series of numerical sensor data112TA-112Tn is stored in the memory 111 in any suitable manner. Forexample, the memory 111 may be configured so that, when the at least onetime series of numerical sensor data 112TA-112Tn is received, the atleast one time series of numerical sensor data 112TA-112Tn iscategorized within the memory. The at least one time series of numericalsensor data 112TA-112Tn may be categorized by one or more of anexcursion 170, by a component 102CA-102Cn and a respective systemparameter 112A-112 n. Where the at least one time series of numericalsensor data 112TA-112Tn is categorized by the excursion 170, the atleast one time series of numerical sensor data 112TA-112Tn iscategorized according to the excursion 170 in which the at least onetime series of numerical sensor data 112TA-112Tn was obtained. Where theat least one time series of numerical sensor data 112TA-112Tn iscategorized by a component 102CA-102Cn, at least one time series ofnumerical sensor data 112TA-112Tn is categorized by the component102CA-102Cn from which the at least one time series of numerical sensordata 112TA-112Tn was obtained. Where the at least one time series ofnumerical sensor data 112TA-112Tn is categorized by the respectivesystem parameter 112A-112 n, the at least one time series of numericalsensor data 112TA-112Tn is categorized by the respective systemparameter 112A-112 n to which the at least one time series of numericalsensor data 112TA-112Tn corresponds (e.g., at least one of (or each of)the at least one time series of numerical sensor data 112TA-112Tncorresponds to a respective system parameter 112A-112 n of the vehiclesystem 102 being monitored).

Referring to FIGS. 1 and 2, the vehicle control module 110 is configuredto generate a graphical representation for the at least one time seriesof numerical sensor data 112TA-112Tn to form at least one analysis image180 of at least one system parameter 112A-112 n. For example, theprocessor 120 of the vehicle control module 110 includes an imagegeneration module 121 that transforms, as illustrated in FIG. 2, the atleast one time series of numerical sensor data 112TA-112TD for therespective system parameter 112A-112D into an analysis image 180A-180D,180CI of at least one system parameter 112A-112D. The vehicle controlmodule 110 (such as the image generation module 121 of the processor120) is configured to access the at least one time series of numericalsensor data 112TA-112Tn from the memory 111, where at least one of (oreach of) the at least one time series of numerical sensor data112TA-112Tn corresponds to a respective system parameter 112A-112 n of avehicle system 102 being monitored. In one aspect, the image generationmodule 121 is configured to generate an analysis image 180A-180D foreach respective system parameter 112A-112D where the image analysisdescribed herein is performed in the individual analysis images180A-180D. In another aspect, the image generation module 121 isconfigured to combine the individual analysis images 180A-180D into acombined analysis image 180CI so that relationships between the systemparameters 112A-112D (e.g., effects on one system parameter resultingfrom a state change of a different system parameter) are graphicallyrepresented or are otherwise revealed to, for example, an operator ofthe vehicle 100. As described herein, the vehicle control module 110 isconfigured to identify relationships between more than one systemparameter 112A-112 n based on the at least one analysis image 180 of atleast one system parameter 112A-112 n. For example, referring also toFIGS. 3A-3D, the combined analysis image 180CI may be a combination ofany number of individual analysis images. For exemplary purposes only,the combined analysis image 180CI2 illustrated in FIG. 3A is acombination of two individual analysis images; the combined analysisimage 180CI4 illustrated in FIG. 3B is a combination of four individualanalysis images; the combined analysis image 180CI7 illustrated in FIG.3C is a combination of seven individual analysis images; and, thecombined analysis image 180CI10 illustrated in FIG. 3D is a combinationof ten individual analysis images. While combined analysis images 180CIare illustrated as including graphical representations of two, four,seven and ten system parameters, in other aspects, any suitable numberof individual analysis images 180 may be combined into a single combinedanalysis image 180CI. As such, the at least one analysis image 180 (suchas the combined analysis image 180CI) of the at least one systemparameter 112A-112 n is common to more than one time series of numericalsensor data 112TA-112Tn. The relationships between the system parameters112A-112 n may be identified by monitoring patterns between the systemparameters 112A-112 n in the at least one analysis image 180. Toidentify the relationships between the system parameters 112A-112 n, theindividual analysis images 180A-180D are generated by the imagegeneration module 121 with a fixed value for the Y axis of theindividual images 180A-180D so that the analysis image(s) (and thetraining image(s) 160 described herein) have the same Y axis scale,where the Y axis represents a sensor 101 value and the X axis representstime.

Still referring to FIGS. 1 and 2 and also to FIGS. 4A-6C, the at leastone analysis image 180 may be generated depending on which portion(s) ofthe excursion 170 are to be analyzed. For example, at least one of (oreach of the at least one time series of numerical sensor data112TA-112Tn corresponds to a whole vehicle excursion 170 where thegraphical representation corresponds to the at least one time series ofnumerical sensor data 112TA-112Tn for the whole vehicle excursion 170.In the case of an aircraft, the excursion 170 is a flight of theaircraft from a departure gate to an arrival gate. In one aspect, ananalysis image 180CIE1-180CIE3 may be generated for the entire excursion170A-170C as illustrated in FIGS. 4A-4C, where the respective analysisimages 180CIE1-180CIE3 presented in FIGS. 4A-4C each represent more thanone system parameter 112A-112D for a single excursion 170A-170C of thevehicle 100. Here, the analysis images 180CIE1-180CIE3 (and the trainingimages 160 as described herein) represent entire flights that may beanalyzed as a whole for faults.

In one aspect, the analysis image 180CIE1-180CIE3 for one or more of theexcursions 170A-170C may be temporally sub-divided into one or moreportions. For example, the analysis image 180CIE1 for excursion 170A inFIG. 4A may be temporally sub-divided into three increments 500A, 500B,500C (e.g., hourly increments, flight stages, or any other predeterminedtemporal division). Here, temporally sub-dividing the analysis image180CIE1-180CIE3 provides for the analysis of specified portions of therespective excursion 170A-170C, which may provide greater faultdetection detail than an analysis of an overall flight, so as tosubstantially eliminate false positives in the anomaly detection. Forexample, false positive anomaly detection may occur where a analysisimages for short duration flight are compared to analysis images for along duration flight (or vice versa) where sub-dividing the analysisimages (and the training images 160 as described herein) provides for amore granular image analysis to reduce the number of false positiveanomaly detection.

In another aspect, as can be seen in FIGS. 6A-6C the at least oneanalysis image 180CIP1-180CIP3 (and the training images 160 as describedherein) may be generated to capture any suitable predetermined timeperiods 610A, 610B before and/or after a predetermined event 600. In oneaspect, the predetermined time periods 610A, 610B may about two minutesbefore and/or after the event 600, while in other aspects thepredetermined time periods may be more or less than about two minutes.In still other aspects, the predetermined time period 610A prior to theevent 600 may be different than the predetermined time period followingthe event 600. For example, system parameter 112A may be the state(e.g., open or closed) of a valve and the event 600 may be the openingand closing of the valve within the environmental system 1252. Thesystem parameter 112B may be a temperature of the environmental system1252. Here it can be seen that the analysis images 180CIP1, 180CIP2 aresubstantially similar while system parameter 112B in analysis image180CIP3 differs, which may be indicative of a fault in system parameter112B. As described above, the analysis images 180CIP1, 180CIP2 may beindicative of a relationship between system parameters 112A, 112B wheresystem parameter 112B graphically responds to the state change of systemparameter 112A as illustrated in FIGS. 6A and 6B. The different shape ofthe curve for system parameter 112B in FIG. 6C may be a departure fromthe relationship between system parameters 112A, 112B as illustrated inFIGS. 6A and 6B and may indicative of a fault in the component 102C (seeFIG. 1) that corresponds with system parameter 112B.

Referring still to FIG. 1, the vehicle control module 110 is configuredto detect anomalous behavior of a component 102C of the vehicle system102 based on the at least one analysis image 180 of at least one systemparameter 112A-112 n. For example, the processor 120 of the vehiclecontrol module 110 includes a deep learning module 122 configured todetect anomalous behavior of the respective system parameter 112A-112 n,with at least one deep learning model 122M, based on the at least oneanalysis image 180 of the at least one system parameter 112A-112 n.Referring also to FIGS. 7A and 7B in one aspect, the at least one deeplearning model 122M includes more than one deep learning model 122MA,122MB configured to detect the anomalous behavior for the component 102Cof the vehicle system 102. Here two deep learning models 122MA, 122MBare illustrated but in other aspects the deep learning module 122 mayinclude any number of deep learning models 122MA-122Mn. As an example,the deep learning module 122 may include a convolutional neural networkdeep learning model 122MA and/or a stacked auto-encoder deep learningmodel 122MB or any other suitable deep learning models. Where the deeplearning module 122 includes more than one deep learning model122MA-122Mn (whether the same type of deep learning model such as, e.g.,more than one convolutional neural network or more than one stackedauto-encoder; or different types of deep learning models such as, acombination of convolutional neural networks and stacked-auto encoders)the deep learning module 122 may be configured to select the deeplearning models 122M for analysis of the at least one analysis image 180depending on a respective predetermined vehicle operating condition 190.In one aspect, the respective predetermined vehicle operating condition190 comprises one or more of an excursion (e.g., in the case of anaircraft, a flight) duration 190A and weather conditions 190B.

Referring to FIGS. 1 and 10, prior to analyzing the at least oneanalysis image 180 the at least one deep learning model 122M is trainedby the vehicle control module 110 (such as by the processor 120). Forexample, the vehicle control module 110 is configured to receive orotherwise obtain at least one historical time series of numerical sensordata 150 (FIG. 10, Block 1000) from at least one historical vehicleexcursion 170H, where the at least one historical time series ofnumerical sensor data 150 corresponds to the respective system parameter112A-112 n of the vehicle system 102 being monitored. The at least onehistorical vehicle excursion 170H may be one or more historicalexcursions 170H for the same vehicle or a number of different vehicleshaving similar characteristics, e.g., a number of different aircrafthaving the same make and model. The at least one historical time seriesof numerical sensor data 150 from the at least one historical vehicleexcursion 170H may be stored in the memory 111 or any other suitablelocation accessible by the vehicle control module 110.

The image generation module 121 is configured to generate a graphicalrepresentation for the at least one historical time series of numericalsensor data 150 (FIG. 10, Block 1010) for a respective historicalvehicle excursion 170H to form at least one training image 160 of the atleast one system parameter 112A-112 n being monitored. The at least onetraining image 160 is generated by the image generation module 121 inthe same manner, as described above with respect to FIGS. 2, 3A-3D,4A-4C, 5, and 6A-6C, that the at least one analysis image 180 isgenerated, again noting that the Y axis (e.g., the sensor value) of theat least one training image 160 has the same scale as Y axis (e.g., thesensor value) of the at least one analysis image 180. As such, in themanner described above, the at least one training image 160 of the atleast one system parameter 112A, 112 n is (where the individual trainingimages are combined into a combined training image) common to more thanone historical time series of numerical sensor data 150 from the atleast one historical vehicle excursion 170H.

The historical nature of the at least one historical excursion 170H andthe respective at least one historical time series of numerical sensordata 150 provides information as to whether the at least one historicaltime series of numerical sensor data 150 and/or the respectivehistorical excursion 170H was/were anomalous or ordinary. The term“anomalous” as used herein means that the sensor data and/or excursionexhibited a deviation from normal operating behavior, which is, ifpersistent, indicative of degraded vehicle system 102 component 102Cperformance and a precursor to fault/failure of the component 102C ofthe vehicle system 102 being monitored. The term “ordinary” as usedherein means that the sensor data and/or excursion exhibited normaloperating characteristics (e.g., no fault/failure) of the component 102Cof the vehicle system 102 being monitored. Using knowledge of whetherthe at least one historical time series of numerical sensor data 150 wasanomalous or ordinary, the vehicle control module 110 is configured tolabel the at least one training image 160 (FIG. 10, Block 1020) of theat least one system parameter 112A, 112 n for the respective historicalvehicle excursion as being one of anomalous 161 (e.g., an anomaloustraining image) or ordinary 162 (e.g., an ordinary training image).

Referring to FIGS. 1, 7A, 7B, and 10, the vehicle control module 110trains the at least one deep learning model 122M (FIG. 10, Block 1030).For example, to train the convolutional neural network deep learningmodel 122MA, the vehicle control module 110 applies the labeled at leastone training image 160 to the convolutional neural network deep learningmodel 122MA. The convolutional neural network deep learning model 122MAscans through each of the labeled at least one training image 160 andidentifies groups of pixels that are similar. Based on the similargroups of pixels the convolutional neural network deep learning model122MA learns to identify the anomalous 161 training images and theordinary 162 training images. As another example, to train the stackedauto-encoder deep learning model 122MB the vehicle control module 110employs the stacked auto-encoder deep learning model 122MB todeconstruct and reconstruct the at least one training image 160. Oncethe model is trained, an analysis image 180 can be deconstructed andthen reconstructed by the model. An amount of error between an originalanalysis image 180 and a respective reconstructed analysis imageidentifies whether the at least one analysis image 180 is anomalous orordinary. For example, if the at least one analysis image 180 representsan ordinary flight (as opposed to anomalous) then the lower the errorbetween the reconstructed analysis image and the respective originalanalysis image 180, the greater the likelihood that the respectiveoriginal analysis image 180 is ordinary. On the other hand, if the atleast one analysis image 180 represents an anomalous flight (as opposedto ordinary) the greater the error between the reconstructed analysisimage and the respective original analysis image 180, the greater thelikelihood that the respective original analysis image 180 is anomalous.An error threshold may be established for the at least one analysisimage 180 so that, when the error in a reconstructed analysis imageexceeds the threshold, the respective analysis image 180 is flagged orotherwise identified as indicating an impending fault in the vehiclesystem 102 component 102C.

Referring to FIGS. 1 and 11, the vehicle control module 110 isconfigured to predict a failure of the component 102C of the vehiclesystem 102 based on the anomalous behavior of the component 102C of thevehicle system 102. For example, during each vehicle excursion 170, theat least one sensor 101 for the component 102C of the vehicle system 102being monitored generates the at least one time series of numericalsensor data 112TA-112Tn (FIG. 11, Block 1110). Prior to, after, and/orduring a vehicle excursion 170, the at least one time series ofnumerical sensor data 112TA-112Tn received from the at least one sensor101 for the component 102C of the vehicle system 102 may be used by theimage generation module 121 where the at least one time series ofnumerical sensor data 112TA-112Tn is transformed into the at least oneanalysis image 180 in the manner described above (FIG. 11, Block 1120).The vehicle control module 110 employs the at least one deep learningmodel 122M of the deep learning module 122 to detect anomalous behaviorof the respective system parameter 112A-112 n based on the at least oneanalysis image 180 of at least one system parameter 112A-112 n (FIG. 11,Block 1130). For example, the convolutional neural network deep learningmodel 122MA (see FIG. 7A) compares the at least one analysis image 180with the knowledge learned from the anomalous 161 and ordinary 162 atleast one training image 160 to determine if the at least one analysisimage 180 is indicative of an impending fault/failure in the component102C of the vehicle system 102 being monitored. As another example, thestacked auto-encoder deep learning model 122MB (see FIG. 7B)deconstructs the at least one analysis image 180 and then reconstructsthe deconstructed at least one analysis image 180 to determine areconstructed input error between the original and reconstructed versionof the at least one analysis image 180. If the reconstructed input erroris over, for example, a predetermined threshold (e.g., about 50% erroror more or less than about 50% error) then the at least one analysisimage 180 is determined by the vehicle control module 110 to beindicative of an impending fault/failure in the component 102C of thevehicle system 102 being monitored.

Referring to FIGS. 1, 7A and 8, an exemplary output of the convolutionalneural network deep learning model 122MA of FIG. 7A (e.g., generatedfrom the at least one time series of numerical sensor data 112TA-112Tnof at least one system parameter 112A-112 n for the component 102C) isillustrated for the component 102C of the vehicle system 102 over anumber of different excursions 170, where each vertical bar represents awhole excursion 170. Here positive values along the Y axis areclassified by the convolutional neural network deep learning model 122MAas being ordinary while the negative values along the Y axis areclassified by the convolutional neural network deep learning model 122MAas being anomalous/faulty. As described above, the convolutional neuralnetwork deep learning model 122MA categorizes each excursion 170 asbeing ordinary or anomalous so that a historical graph, such as the oneillustrated in FIG. 8, may be presented to an operator through the userinterface 125. Considering excursions from about excursion number 1 toabout excursion number 10, the output of the convolutional neuralnetwork deep learning model 122MA indicates that the analysis image 180began to deviate from being an ordinary analysis image 180NRM at aboutexcursion 5 and exhibited characteristics of an anomalous analysis image180ABN (noting the difference between system parameter 112B in theordinary analysis image 180NRM and the anomalous analysis image 180ABN).Based on the repeated negative values of the analysis image 180 fromabout excursion 6 to about excursion 9, the vehicle control module 110may predict a fault/failure in the component 102C which is shown in FIG.8 as occurring at about excursion 10. The vehicle control module 110 maybe configured to cause the user interface 125 to present/display theoutput of the convolutional neural network deep learning model 122MA onthe user interface 125 as an indication of anomalous behavior 189 in thecomponent 102C for the prediction 189P of the fault/failure to theoperator (FIG. 11, Block 1140), where when degraded performance of thecomponent 102C is illustrated in the deep learning model output theoperator may perform preventative maintenance on the component 102C(FIG. 11, Block 1150) prior to a component 102C fault/failure occurring.

Referring to FIGS. 1, 7B, and 9A-9C, an exemplary output of the stackedauto-encoder deep learning model 122MB of FIG. 7B (e.g., generated fromthe at least one time series of numerical sensor data 112TA-112Tn of atleast one system parameter 112A-112 n for the component 102C) isillustrated for the component 102C of the vehicle system 102 over anumber of different excursions 170. In FIG. 9A each vertical barrepresents a whole excursion. In FIG. 9B each vertical bar represents aselected portion of the excursion (e.g., a selected flight phase such aspower on through climb). FIG. 9C illustrates a combination of FIGS. 9Aand 9B where the resulting graph may eliminate false positives caused byshort flights.

As can be seen in FIG. 9A a larger reconstructed input error for aflight may be indicative of an impending failure in the correspondingcomponent 102C. Here, for exemplary purposes only reconstructed inputerror values of over about 100 (in other aspects any suitablereconstructed input error value may be used) may be indicative of animpending failure such that sustained numbers of excursions 170exceeding the reconstructed input error value of about 100 (see e.g.,the excursions from about excursion number 56 to about excursion number67 and the excursions from about excursion number 96 to about excursionnumber 99) is predictive of the fault/failure of the component 102C (seethe failure of the component at about excursion numbers 67 and 99).However, there may be instances where an excursion may be indicative ofa fault due to characteristics of the excursion itself (e.g., a durationof the excursion or a duration of a portion of the excursion). Forexample, the excursion indicated at about excursion number 35 in FIG. 9Ais shown as a spike (e.g., a large increase) in the reconstructed inputerror (e.g., having a reconstructed input error of about 200) however,further analysis of the excursion (as described below with respect toFIG. 9B) indicates that the spike in the reconstructed input error shownin the graph was due to the excursion having a short duration comparedto the other excursions shown in the graph (e.g., noting that a numberof excursions prior to and following the spike are below thereconstructed input error value of about 100; see also the excursionshown at about excursion number 74 which exhibits a reconstructed inputerror value over about 100 due to a longer tail flight phase).

To determine whether spikes in the reconstructed input error for a wholeexcursion are false positives, the excursions may be subdivided intoflight phases in the manner described with respect to, e.g., FIG. 5.FIG. 9B is a graphical illustration of the reconstructed input error forthe select excursion (in this case a flight) phase of the vehicle 100from power on through climb. Based on an image analysis of selectedflight phase, it can be seen in FIG. 9B that the excursionscorresponding to the vertical bars at about excursion numbers 35 and 74in FIG. 9A are ordinary excursions. It can also be seen in FIG. 9B thatthe excursions corresponding to the vertical bars at about excursionnumbers 67 and 99 in FIG. 9A are confirmed to be anomalous excursionsindicative of a component 102C fault/failure.

As noted above, the graphs illustrated in FIGS. 9A and 9B can becombined into a single graph illustrated in FIG. 9C, which may eliminatefalse positive failure indications. For example, referring to graphregions A and B in FIG. 9C, it can be seen that the full flight imageanalysis is accompanied by a flight phase analysis that indicates alarge discrepancy between the respective reconstructed input error forthe respective excursion numbers. The large discrepancy between the fullflight image analysis and the flight phase image analysis for the sameexcursion number is indicative of a false positive indication offault/failure. On the other hand, referring to graph regions C and D inFIG. 9C, where the reconstructed input error for both the full flightimage analysis and the flight phase image analysis substantiallycoincide (e.g., the values are both above for example, about 100, whichmay be a threshold value for indicating fault/failure) the graphindicates/predicts a fault/failure of the component 102C.

In a manner similar to that described above with respect to theconvolutional neural network deep learning model 122MA (see FIG. 7A),the vehicle control module 110 may be configured to cause the userinterface 125 to present/display the output (e.g., one or more of thegraphs illustrated in FIGS. 9A-9C) of the stacked auto-encoder deeplearning model 122MB (FIG. 7B) on the user interface 125 as anindication of anomalous behavior 189 in the component 102C for theprediction 189P of the failure to the operator (FIG. 11, Block 1140),where when degraded performance of the component 102C is illustrated inthe deep learning model output the operator may perform preventativemaintenance on the component 102C (FIG. 11, Block 1150) prior to acomponent 102C fault/failure occurring.

Referring to FIG. 1, the vehicle control module 110 may perform theabove-described fault detection and failure prediction analysis atvehicle 100 startup and/or prior to any suitable specified operation ofthe vehicle 100 (e.g., for example, dropping an anchor, docking with aspace station, operating a robotic arm of the vehicle, etc.). In otheraspects, the vehicle control module 110 may perform the above-describedfault detection and fault prediction analysis at vehicle 100 shutdown.For example, the vehicle fault detection system 199 may include avehicle interlock 197 coupled with the vehicle control module 110 andone or more of the vehicle systems 102. The vehicle interlock 197 isconfigured to prevent an operation (corresponding to a vehicle system102 coupled to the interlock 197) of the vehicle 100 based on adetection of the anomalous behavior that is indicative of a component102C fault/failure. Here, if the vehicle control module 110 detectsanomalous behavior of a vehicle system 102 component 102C the interlock197 may prevent the operation of the vehicle system component 102C. Forexample, if anomalous behavior is detected for a robotic arm of aspacecraft (that includes the fault detection system 199 describedherein) the vehicle interlock 197 may prevent the operation of therobotic arm to allow for the performance of preventative maintenance. Inother aspects, the vehicle interlock 197 may be configured to providelimited/restricted use of the vehicle component 102C when anomalousbehavior of the vehicle system 102 component 102C is detected by thevehicle fault detection system 199.

Referring to FIGS. 12 and 13, examples of the present disclosure may bedescribed in the context of aircraft manufacturing and service method1300 as shown in FIG. 13. In other aspects, the examples of the presentdisclosure may be applied in any suitable industry, such as e.g.,automotive, maritime, aerospace, etc. as noted above. With respect toaircraft manufacturing, during pre-production, illustrative method 1300may include specification and design (block 1310) of aircraft 100A andmaterial procurement (block 1320). During production, component andsubassembly manufacturing (block 1330) and system integration (block1340) of aircraft 100A may take place. Thereafter, aircraft 100A may gothrough certification and delivery (block 1350) to be placed in service(block 1360). While in service, aircraft 100A may be scheduled forroutine maintenance and service (block 1370). Routine maintenance andservice may include modification, reconfiguration, refurbishment, etc.of one or more systems of aircraft 100A which may include and/or befacilitated by the fault determination described herein.

Each of the processes of illustrative method 1300 may be performed orcarried out by a system integrator, a third party, and/or an operator(e.g., a customer). For the purposes of this description, a systemintegrator may include, without limitation, any number of aircraftmanufacturers and major-system subcontractors; a third party mayinclude, without limitation, any number of vendors, subcontractors, andsuppliers; and an operator may be an airline, leasing company, militaryentity, service organization, and so on.

The apparatus(es), system(s), and method(s) shown or described hereinmay be employed during any one or more of the stages of themanufacturing and service method 1300. For example, components orsubassemblies corresponding to component and subassembly manufacturing(block 1330) may be fabricated or manufactured in a manner similar tocomponents or subassemblies produced while aircraft 100A is in service(block 1360). Similarly, one or more examples of the apparatus or methodrealizations, or a combination thereof, may be utilized, for example andwithout limitation, while aircraft 100A is in service (block 1360)and/or during maintenance and service (block 1370).

The following are provided in accordance with the aspects of the presentdisclosure:

A1. A vehicle fault detection system comprising:

at least sensor configured for coupling with a vehicle system;

a vehicle control module coupled to the at least one sensor, the vehiclecontrol module being configured to

receive at least one time series of numerical sensor data from the atleast one sensor, at least one of (or each of) the at least one timeseries of numerical sensor data corresponds to a respective systemparameter of the vehicle system being monitored,

generate a graphical representation for the at least one time series ofnumerical sensor data to form an analysis image of at least one systemparameter, and

detect anomalous behavior of a component of the vehicle system based onthe analysis image of at least one system parameter; and

a user interface coupled to the vehicle control module, the userinterface being configured to present to an operator an indication ofthe anomalous behavior for the component of the vehicle system.

A2. The vehicle fault detection system of paragraph A1, wherein thevehicle is one of an automotive vehicle, a maritime vehicle, and anaerospace vehicle.

A3. The vehicle fault detection system of paragraph A1, wherein thevehicle is an aircraft.

A4. The vehicle fault detection system of paragraph A1, wherein theanalysis image of at least one system parameter is common to more thanone time series of numerical sensor data.

A5. The vehicle fault detection system of paragraph A1, wherein thevehicle control module includes a deep learning module including atleast one deep learning model configured to detect the anomalousbehavior for the component of the vehicle system.

A6. The vehicle fault detection system of paragraph A5, wherein the atleast one deep learning model includes more than one deep learning modelconfigured to detect the anomalous behavior for the component of thevehicle system depending on a respective predetermined vehicle operatingcondition.

A7. The vehicle fault detection system of paragraph A6, wherein therespective predetermined vehicle operating condition comprises one ormore of a flight duration and weather conditions.

A8. The vehicle fault detection system of paragraph A5, wherein thevehicle control module is configured to train the at least one deeplearning model by:

receiving at least one historical time series of numerical sensor datafrom at least one historical vehicle excursion, the at least onehistorical time series of numerical sensor data corresponds to therespective system parameter of the vehicle system being monitored,

generating a graphical representation for the at least one historicaltime series of numerical sensor data for a respective historical vehicleexcursion to form a training image of at least one system parameter, and

labeling the training image of at least one system parameter for therespective historical vehicle excursion as being one of anomalous orordinary.

A9. The vehicle fault detection system of paragraph A8, wherein thetraining image of at least one system parameter is common to more thanone historical time series of numerical sensor data from the at leastone historical vehicle excursion.

A10. The vehicle fault detection system of paragraph A8, wherein atleast one of (or each of) the at least one historical vehicle excursionis a flight of the vehicle.

A11. The vehicle fault detection system of paragraph A5, wherein the atleast one deep learning model comprises a convolutional neural network.

A12. The vehicle fault detection system of paragraph A5, wherein the atleast one deep learning model comprises a stacked auto-encoder.

A13. The vehicle fault detection system of paragraph A1, wherein atleast one of (or each of) the at least one time series of numericalsensor data corresponds to a whole vehicle excursion where the graphicalrepresentation corresponds to the at least one time series of numericalsensor data for the whole vehicle excursion.

A14. The vehicle fault detection system of paragraph A1, wherein thevehicle control module is configured to identify relationships betweenmore than one system parameter based on the analysis image of at leastone system parameter.

A15. The vehicle fault detection system of paragraph A1, furthercomprising a vehicle interlock coupled with the vehicle control module,the vehicle interlock being configured to prevent an operation of thevehicle based on a detection of the anomalous behavior.

A16. The vehicle fault detection system of paragraph A1, wherein thevehicle control module is further configured to predict a failure of thecomponent of the vehicle system based on the anomalous behavior of thecomponent of the vehicle system and the user interface is furtherconfigured to present prediction of the failure to the operator.

B1. A vehicle fault detection system comprising:

a memory;

at least one sensor coupled to the memory, the at least one sensor beingconfigured to generate at least one time series of numerical sensor datafor a respective system parameter of a vehicle system being monitored;

a vehicle control module coupled to the memory, the vehicle controlmodule being configured to transform the at least one time series ofnumerical sensor data for the respective system parameter into ananalysis image of at least one system parameter and detect, with atleast one deep learning model, anomalous behavior of the respectivesystem parameter based on the analysis image of at least one systemparameter; and

a user interface coupled to the vehicle control module, the userinterface being configured to present to an operator an indication ofthe anomalous behavior of the respective system parameter.

B2. The vehicle fault detection system of paragraph B1, wherein thevehicle control module is configured to access the at least one timeseries of numerical sensor data from the memory, at least one of (oreach of) the at least one time series of numerical sensor datacorresponds to a respective system parameter of a vehicle system beingmonitored.

B3. The vehicle fault detection system of paragraph B1, wherein thevehicle is one of an automotive vehicle, a maritime vehicle, and anaerospace vehicle.

B4. The vehicle fault detection system of paragraph B1, wherein thevehicle is an aircraft.

B5. The vehicle fault detection system of paragraph B1, wherein theanalysis image of at least one system parameter is common to more thanone time series of numerical sensor data.

B6. The vehicle fault detection system of paragraph B1, wherein thevehicle control module includes a deep learning module including theleast one deep learning model.

B7. The vehicle fault detection system of paragraph B1, wherein the atleast one deep learning model includes more than one deep learning modelconfigured to detect the anomalous behavior for a component of thevehicle system depending on a respective predetermined vehicle operatingcondition.

B8. The vehicle fault detection system of paragraph B7, wherein therespective predetermined vehicle operating condition comprises one ormore of a flight duration and weather conditions.

B9. The vehicle fault detection system of paragraph B1, wherein thevehicle control module is configured to train the at least one deeplearning model by:

receiving at least one historical time series of numerical sensor datafrom at least one historical vehicle excursion, the at least onehistorical time series of numerical sensor data corresponds to therespective system parameter of the vehicle system being monitored,

generating a graphical representation for the at least one historicaltime series of numerical sensor data for a respective historical vehicleexcursion to form a training image of at least one system parameter, and

labeling the training image of at least one system parameter for therespective historical vehicle excursion as being one of anomalous orordinary.

B10. The vehicle fault detection system of paragraph B9, wherein thetraining image of at least one system parameter is common to more thanone historical time series of numerical sensor data from the at leastone historical vehicle excursion.

B11. The vehicle fault detection system of paragraph B9, wherein atleast one of (or each the at least one historical vehicle excursion is aflight of the vehicle.

B12. The vehicle fault detection system of paragraph B1, wherein the atleast one deep learning model comprises a convolutional neural network.

B13. The vehicle fault detection system of paragraph B1, wherein the atleast one deep learning model comprises a stacked auto-encoder.

B14. The vehicle fault detection system of paragraph B1, wherein atleast one of (or each of) the at least one time series of numericalsensor data corresponds to a whole vehicle excursion and the analysisimage corresponds to the at least one time series of numerical sensordata for the whole vehicle excursion.

B15. The vehicle fault detection system of paragraph B1, wherein thevehicle control module is configured to identify relationships betweenmore than one system parameter based on the analysis image of at leastone system parameter.

B16. The vehicle fault detection system of paragraph B1, furthercomprising a vehicle interlock coupled with the vehicle control module,the vehicle interlock being configured to prevent an operation of thevehicle based on a detection of the anomalous behavior.

B17. The vehicle fault detection system of paragraph B1, wherein thevehicle control module is further configured to predict a failure of acomponent of the vehicle system based on the anomalous behavior of therespective system parameter and the user interface is further configuredto present prediction of the failure to the operator.

C1. A method for vehicle fault detection, the method comprising:

generating, with at least one sensor coupled to a vehicle system, atleast one time series of numerical sensor data for a respective systemparameter of the vehicle system being monitored;

transforming, with a vehicle control module coupled to the at least onesensor, the at least one time series of numerical sensor data for therespective system parameter into an analysis image of at least onesystem parameter;

detecting, with at least one deep learning model of the vehicle controlmodule, anomalous behavior of the respective system parameter based onthe analysis image of at least one system parameter; and

displaying, on a user interface coupled to the vehicle control module,an indication of the anomalous behavior of the respective systemparameter.

C2. The method of paragraph C1, wherein at least one of (or each of) theat least one time series of numerical sensor data corresponds to arespective system parameter of a vehicle system being monitored.

C3. The method of paragraph C1, wherein the vehicle is one of anautomotive vehicle, a maritime vehicle, and an aerospace vehicle.

C4. The method of paragraph C1, wherein the vehicle is an aircraft.

C5. The method of paragraph C1, wherein the analysis image of at leastone system parameter is common to more than one time series of numericalsensor data.

C6. The method of paragraph C1, wherein the at least one deep learningmodel includes respective deep learning models corresponding todifferent predetermined vehicle operating conditions and the anomalousbehavior for a component of the vehicle system is detected with therespective deep learning models depending on the predetermined vehicleoperating condition.

C7. The method of paragraph C6, wherein the respective predeterminedvehicle operating condition comprises one or more of a flight durationand weather conditions.

C8. The method of paragraph C1, further comprising training the at leastone deep learning model, with the vehicle control module, by:

receiving at least one historical time series of numerical sensor datafrom at least one historical vehicle excursion, the at least onehistorical time series of numerical sensor data corresponds to therespective system parameter of the vehicle system being monitored,

generating a graphical representation for the at least one historicaltime series of numerical sensor data for a respective historical vehicleexcursion to form a training image of at least one system parameter, and

labeling the training image of at least one system parameter for therespective historical vehicle excursion as being one of anomalous orordinary.

C9. The method of paragraph C8, wherein the training image of at leastone system parameter is common to more than one historical time seriesof numerical sensor data from the at least one historical vehicleexcursion.

C10. The method of paragraph C8, wherein at least one of (or each of)the at least one historical vehicle excursion is a flight of thevehicle.

C11. The method of paragraph C1, wherein the at least one deep learningmodel comprises a convolutional neural network.

C12. The method of paragraph C1, wherein the at least one deep learningmodel comprises a stacked auto-encoder.

C13. The method of paragraph C1 wherein at least one of (or each of) theat least one time series of numerical sensor data corresponds to a wholevehicle excursion and the analysis image corresponds to the at least onetime series of numerical sensor data for the whole vehicle excursion.

C14. The method of paragraph C1, further comprising identifying, withthe vehicle control module, relationships between more than one systemparameter based on the analysis image of at least one system parameter.

C15. The method of paragraph C1, further comprising preventing anoperation of the vehicle, with a vehicle interlock coupled with thevehicle control module, based on a detection of the anomalous behavior.

C16. The method of paragraph C1, further comprising:

predicting, with the vehicle control module, a failure of a component ofthe vehicle system based on the anomalous behavior of the respectivesystem parameter; and

displaying, on the user interface, prediction of the failure.

In the figures, referred to above, solid lines, if any, connectingvarious elements and/or components may represent mechanical, electrical,fluid, optical, electromagnetic, wireless and other couplings and/orcombinations thereof. As used herein, “coupled” means associateddirectly as well as indirectly. For example, a member A may be directlyassociated with a member B, or may be indirectly associated therewith,e.g., via another member C. It will be understood that not allrelationships among the various disclosed elements are necessarilyrepresented. Accordingly, couplings other than those depicted in thedrawings may also exist. Dashed lines, if any, connecting blocksdesignating the various elements and/or components represent couplingssimilar in function and purpose to those represented by solid lines;however, couplings represented by the dashed lines may either beselectively provided or may relate to alternative examples of thepresent disclosure. Likewise, elements and/or components, if any,represented with dashed lines, indicate alternative examples of thepresent disclosure. One or more elements shown in solid and/or dashedlines may be omitted from a particular example without departing fromthe scope of the present disclosure. Environmental elements, if any, arerepresented with dotted lines. Virtual (imaginary) elements may also beshown for clarity. Those skilled in the art will appreciate that some ofthe features illustrated in the figures, may be combined in various wayswithout the need to include other features described in the figures,other drawing figures, and/or the accompanying disclosure, even thoughsuch combination or combinations are not explicitly illustrated herein.Similarly, additional features not limited to the examples presented,may be combined with some or all of the features shown and describedherein.

In FIGS. 10, 11, and 13, referred to above, the blocks may representoperations and/or portions thereof and lines connecting the variousblocks do not imply any particular order or dependency of the operationsor portions thereof. Blocks represented by dashed lines indicatealternative operations and/or portions thereof. Dashed lines, if any,connecting the various blocks represent alternative dependencies of theoperations or portions thereof. It will be understood that not alldependencies among the various disclosed operations are necessarilyrepresented. FIGS. 10, 11, and 13 and the accompanying disclosuredescribing the operations of the method(s) set forth herein should notbe interpreted as necessarily determining a sequence in which theoperations are to be performed. Rather, although one illustrative orderis indicated, it is to be understood that the sequence of the operationsmay be modified when appropriate. Accordingly, certain operations may beperformed in a different order or substantially simultaneously.Additionally, those skilled in the art will appreciate that not alloperations described need be performed.

In the following description, numerous specific details are set forth toprovide a thorough understanding of the disclosed concepts, which may bepracticed without some or all of these particulars. In other instances,details of known devices and/or processes have been omitted to avoidunnecessarily obscuring the disclosure. While some concepts will bedescribed in conjunction with specific examples, it will be understoodthat these examples are not intended to be limiting.

Unless otherwise indicated, the terms “first,” “second,” etc. are usedherein merely as labels, and are not intended to impose ordinal,positional, or hierarchical requirements on the items to which theseterms refer. Moreover, reference to, e.g., a “second” item does notrequire or preclude the existence of, e.g., a “first” or lower-numbereditem, and/or, e.g., a “third” or higher-numbered item.

Reference herein to “one example” means that one or more feature,structure, or characteristic described in connection with the example isincluded in at least one implementation. The phrase “one example” invarious places in the specification may or may not be referring to thesame example.

As used herein, a system, apparatus, structure, article, element,component, or hardware “configured to” perform a specified function isindeed capable of performing the specified function without anyalteration, rather than merely having potential to perform the specifiedfunction after further modification. In other words, the system,apparatus, structure, article, element, component, or hardware“configured to” perform a specified function is specifically selected,created, implemented, utilized, programmed, and/or designed for thepurpose of performing the specified function. As used herein,“configured to” denotes existing characteristics of a system, apparatus,structure, article, element, component, or hardware which enable thesystem, apparatus, structure, article, element, component, or hardwareto perform the specified function without further modification. Forpurposes of this disclosure, a system, apparatus, structure, article,element, component, or hardware described as being “configured to”perform a particular function may additionally or alternatively bedescribed as being “adapted to” and/or as being “operative to” performthat function.

Different examples of the apparatus(es) and method(s) disclosed hereininclude a variety of components, features, and functionalities. Itshould be understood that the various examples of the apparatus(es),system(s), and method(s) disclosed herein may include any of thecomponents, features, and functionalities of any of the other examplesof the apparatus(es) and method(s) disclosed herein in any combination,and all of such possibilities are intended to be within the scope of thepresent disclosure.

Many modifications of examples set forth herein will come to mind to oneskilled in the art to which the present disclosure pertains having thebenefit of the teachings presented in the foregoing descriptions and theassociated drawings.

Therefore, it is to be understood that the present disclosure is not tobe limited to the specific examples illustrated and that modificationsand other examples are intended to be included within the scope of theappended claims. Moreover, although the foregoing description and theassociated drawings describe examples of the present disclosure in thecontext of certain illustrative combinations of elements and/orfunctions, it should be appreciated that different combinations ofelements and/or functions may be provided by alternative implementationswithout departing from the scope of the appended claims. Accordingly,parenthetical reference numerals in the appended claims are presentedfor illustrative purposes only and are not intended to limit the scopeof the claimed subject matter to the specific examples provided in thepresent disclosure.

What is claimed is:
 1. A vehicle fault detection system comprising: atleast one sensor configured for coupling with a vehicle system; avehicle control module that is coupled to the at least one sensor andthat includes a deep learning module, the vehicle control module beingconfigured to obtain at least one historical time series of numericalsensor data that corresponds to a respective system parameter of thevehicle system being monitored, generate more than one labeled trainingimage from the at least one historical time series of numerical sensordata, the more than one labeled training image includes graphicalrepresentations of the at least one historical time series of numericalsensor data that are labeled as anomalous training images and ordinarytraining images, train the deep learning module with the more than onelabeled training image so that the deep learning module learns toidentify the anomalous training images and the ordinary training images,receive at least one time series of numerical sensor data from the atleast one sensor, at least one of the at least one time series ofnumerical sensor data corresponds to the respective system parameter ofthe vehicle system being monitored, generate an analysis image thatincludes a graphical representation for the at least one time series ofnumerical sensor data corresponding to at least one system parameter,and detect, with the deep learning module, anomalous behavior of acomponent of the vehicle system based on an image graphic comparisonanalysis of the graphical representation of the at least one systemparameter with the knowledge learned by the training of the deeplearning module with the anomalous training images and the ordinarytraining images; and a user interface coupled to the vehicle controlmodule, the user interface being configured to present to an operator anindication of the anomalous behavior for the component of the vehiclesystem.
 2. The vehicle fault detection system of claim 1, wherein thevehicle is one of an automotive vehicle, a maritime vehicle, and anaerospace vehicle.
 3. The vehicle fault detection system of claim 1,wherein the analysis image of at least one system parameter is agraphical representation of more than one time series of numericalsensor data.
 4. The vehicle fault detection system of claim 1, whereinthe deep learning module includes at least one deep learning modelhaving a neural network configured to detect the anomalous behavior forthe component of the vehicle system.
 5. The vehicle fault detectionsystem of claim 4, wherein the at least one deep learning model includesmore than one deep learning model configured to detect the anomalousbehavior for the component of the vehicle system depending on arespective predetermined vehicle operating condition.
 6. The vehiclefault detection system of claim 1, wherein the vehicle control module isconfigured to combine the analysis image of one of the at least onesystem parameter with an analysis image of at least another of the atleast one system parameter to graphically represent relationshipsbetween more than one system parameter based on the analysis image of atleast one system parameter.
 7. A vehicle fault detection systemcomprising: a memory; and a vehicle control module coupled to thememory, the vehicle control module includes at least one deep learningmodule and is configured to obtain at least one historical time seriesof numerical sensor data that corresponds to a respective systemparameter of a vehicle system being monitored, transform the at leastone historical time series of numerical sensor data into more than onelabeled training image, the more than one labeled training imageincludes graphical representations of the at least one historical timeseries of numerical sensor data that are labeled as anomalous trainingimages and ordinary training images, train the at least one deeplearning module with the more than one labeled training image so thatthe at least one deep learning module learns to identify the anomaloustraining images and the ordinary training images, transform at least onetime series of numerical sensor data for the respective system parameterof the vehicle system being monitored into an analysis image thatincludes a graphical representation of at least one system parameter,and detect, with the at least one deep learning model, anomalousbehavior of the respective system parameter based on an image graphiccomparison analysis of the graphical representation of the at least onesystem parameter with the knowledge learned by the training of the atleast one deep learning module with the anomalous training images andthe ordinary training images; wherein the at least one time series ofnumerical sensor data is generated by at least one sensor coupled to thememory and an indication of the anomalous behavior of the respectivesystem parameter is presented to an operator through a user interfacecoupled to the vehicle control module.
 8. The vehicle fault detectionsystem of claim 7, wherein the vehicle control module is configured toaccess the at least one time series of numerical sensor data from thememory, at least one of the at least one time series of numerical sensordata corresponds to a respective system parameter of a vehicle systembeing monitored.
 9. The vehicle fault detection system of claim 7,wherein the at least one deep learning model includes more than one deeplearning model configured to detect the anomalous behavior for acomponent of the vehicle system depending on a respective predeterminedvehicle operating condition.
 10. The vehicle fault detection system ofclaim 7, wherein the vehicle control module is configured to train theat least one deep learning model by: receiving at least one historicaltime series of numerical sensor data from at least one historicalexcursion, the at least one historical time series of numerical sensordata corresponds to the respective system parameter of the vehiclesystem being monitored, generating a graphical representation for the atleast one historical time series of numerical sensor data for arespective historical vehicle excursion to form a training image of atleast one system parameter, and labeling the training image of at leastone system parameter for the respective historical vehicle excursion asbeing one of anomalous or ordinary.
 11. The vehicle fault detectionsystem of claim 10, wherein the training image of at least one systemparameter is a graphical representation of more than one historical timeseries of numerical sensor data from the at least one historical vehicleexcursion.
 12. The vehicle fault detection system of claim 10, whereinat least one of the at least one historical vehicle excursion is aflight of the vehicle.
 13. The vehicle fault detection system of claim7, wherein the at least one deep learning model comprises aconvolutional neural network.
 14. The vehicle fault detection system ofclaim 7, wherein the at least one deep learning model comprises astacked auto-encoder.
 15. The vehicle fault detection system of claim 7,further comprising a vehicle interlock coupled with the vehicle controlmodule, the vehicle interlock being configured to prevent an operationof the vehicle based on a detection of the anomalous behavior.
 16. Amethod for vehicle fault detection, the method comprising: obtaining,with a vehicle control module, at least one historical time series ofnumerical sensor data that corresponds to a respective system parameterof a vehicle system being monitored, the vehicle control moduleincluding at least one deep learning module; generating more than onelabeled training image from the at least one historical time series ofnumerical sensor data, the more than one labeled training image includesgraphical representations of the at least one historical time series ofnumerical sensor data that are labeled as anomalous training images andordinary training images; training, with the vehicle control module, theat least one deep learning module with the more than one labeledtraining image so that the at least one deep learning module learns toidentify the anomalous training images and the ordinary training images,with the vehicle control module, receiving, from at least one sensorcoupled to both a vehicle system and the vehicle control module, atleast one time series of numerical sensor data for a respective systemparameter of the vehicle system being monitored; transforming, with thevehicle control module, the at least one time series of numerical sensordata for the respective system parameter into an analysis image thatincludes a graphical representation of at least one system parameter;detecting, with at least one deep learning model of the vehicle controlmodule, anomalous behavior of the respective system parameter based onan image graphic comparison analysis of the graphical representation ofthe at least one system parameter; and facilitating, via a userinterface coupled to the vehicle control module, a display of anindication of the anomalous behavior of the respective system parameter.17. The method of claim 16, wherein at least one of the at least onetime series of numerical sensor data corresponds to a respective systemparameter of a vehicle system being monitored.
 18. The method of claim16, wherein the analysis image of at least one system parameter is agraphical representation of more than one time series of numericalsensor data.
 19. The method of claim 16, wherein the at least one deeplearning model includes respective deep learning models corresponding todifferent predetermined vehicle operating conditions and the anomalousbehavior for a component of the vehicle system is detected with therespective deep learning models depending on the predetermined vehicleoperating condition.
 20. The method of claim 16, further comprisingcombining, with the vehicle control module, the analysis image of therespective system parameter with another analysis image of anothersystem parameter to graphically represent relationships between morethan one system parameter based on the analysis image of at least onesystem parameter.